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Abstract 

University education has been massified in South Africa and this has disrupted conventions and practices of 
assessment. Universities are perceived as promoting complex achievements and complex skills through rigorous 
curriculum specifications and assessment instruments. Due to experiences in high school and the novelty of 
university education, students have begun to enquire about the ‘scope of the paper.’ This enquiry has tended to 
destabilise the reliability of assessments and judgments in universities in South Africa. Grades and symbols 
certified by some universities have become both indeterminate and ambiguous. A qualitative, quantitative design 
was adopted to identify and interrogate university research participants’ views on experiences about 
examinations at university level. In the process, the meanings that first-year degree-level test-takers ascribed to 
the concept of ‘the scope of the paper’ were unpacked. It was determined which aspects of the test scope 
students had studied in order to prepare for tests. Inferences were made about the concepts that the students 
marginalised because they were perceived to be ‘out of the scope’ of the examination. A holistic and broad 
educational experience for university students is recommended in spite of the daunting numbers enrolled in 
certain university courses. 
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Out of uniform 

The research reported in this article sought to problematize a widespread phenomenon observed in South African 
educational sites, ranging from high schools to public universities - a phenomenon that was designated as ‘the 
scope of the paper.’ The uniform school system and out-of-uniform university education systems set ‘significant’ 
tests and examinations for a variety of purposes: placement, selection, skills competency profiling and 
assessment of learning, and these tests are purportedly ‘highly reliable’ (Knight, 2002). In the official parlance of 
South African educational discourse, it can be averred that the intended outcomes have been reached by the 
learners. 

Generally, curriculum statements explicitly state the grade-appropriate assessments that are 
administered at each stage. Generally too, students perceive the high stakes significance of such assessments and 
demonstrate an anxiety to ‘succeed against all odds’ in such tests. Before the actual examinations, it has been 
observed students demonstrate their anxiety to succeed by asking their lecturers to explain ‘the scope of the 
examination’. As the test dates approach, students, both weak and strong, tend to ask this inevitable question The 
research reported below interrogated the epistemological, political and pedagogical implications of this ‘test- 
wise’ phenomenon as it seemed to imbricate notions of negative washback (Anderson et al, 1995:46) and 
examination (under)preparedness in South African sites of higher education. In essence, the investigation sought 
to trace the skills and understandings that are valued by students and universities in South Africa, skills related to 
specific disciplines that include, amongst others, reflexivity, engagement in communities of academic practice 
and strategic thinking. 

Research context and problem statement 

This segment provides an excursus into the fundamental conceptual tools of analysis: the curriculum, assessment 
of learning and the power of tests. In tandem, the segment historicizes student access to and anxiety over the 
construct complexity of assessment instruments at three universities in South Africa. Examination taking 
students at Rhodes University, the University of Limpopo and North-West University naturalistically asked their 
lecturers what the scope of the question paper was. With this naturalistic and anecdotal query in mind, this study 
specifically focused on North-West University in order to establish the multiple meanings conveyed by such a 
question. Curriculum quality in South African universities is critical in so far as it entails traditions of 
consistency, coherence, content depth and breadth and progression from one level to higher intended outcomes. 
As Knight (2002:276) observes, ‘the award of a degree should attest that graduates have achieved what the 
[university] curriculum specifies, with the added implication that judgments of achievement are based on reliable 
assessments of student learning.’ This observation neatly links with the precondition that university assessment 
must exhibit demonstrable fidelity to the curriculum. University assessment ought to be pitched at sufficient test 
construct complexity, a condition that subsumes that university education strives to confer some degree of 
abstraction, generalisation and transferability of acquired skills in the specific discipline/s for which the degree 
certificate is awarded. 
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Twenty years after the democratic dispensation in South Africa, curriculum policy and assessment 
practices have been characterised by incessant change. The adoption of outcomes-based education (OBE) in 
1996 was driven largely by a redress agenda where issues of equity and access were justifiably and significantly 
privileged. It is not an overstatement that school assessment tools and practices developed and administered until 
Curriculum 2005 were apparently anchored on compensatory practices. Consequently, a majority of candidates 
were ‘coached from the margins’ and given access to test papers in order to statistically increase pass rates 
across schools (Balfour, 2012). In addition to pre-test papers that were developed and actively taught and revised 
in schools, winter and summer schools were purposely encouraged in order to give candidates greater practice on 
the ‘scope of the question paper’ that candidates would sit for matriculation examinations. South African 
Broadcasting Services (SABC1), a quasi-government radio and television channel, disseminated examination 
‘scope’ materials for individual subjects, and so did The Star , a daily newspaper that also provided inserts with 
worked examples and sample answers for ‘typical scope questions’ that were likely to be part of the examination 
proper. 

In a nutshell, several agents - including subject specialists in the schools - participated in the 
development and dissemination of ‘examination scope’ materials. Whereas the innocent intention was to redress 
past injustices and adequately prepare test-takers for what to expect in the examinations, in retrospect, the drill¬ 
like practices and rehearsals de-emphasised interrogative, evaluative and critical reflective skills in the test- 
takers (Fairclough, 2006; Gee, 2005). The teaching specialists in high schools focused invariably on examination 
prototypical questions and they honed examination scope competencies in the test-takers. The table below shows 
the statistical increases in test-takers’ pass rates at national level from 2009 to 2013.The significant increase in 
pass rates suggests the impact of ‘coaching’ practices in schools and is used here for illustrative purposes. 


Annual increases in pass rates at Matriculation level, SA, 2009-2013. 


Year 

Total candidates 

Recorded passes 

% increase 

2009 

559 000 

60.6% 

N/A 

2010 

537 543 

67.8% 

6.8 

2011 

496 090 

70.2% 

2.4 

2012 

623 897 

73.9% 

3.7 

2013 

654 723 

78.2% 

4.3 


The table above gives only the statistics as disseminated by the Department of Education (DoE) and it 
does not supply details on the statistics of test-takers who had university admission exemption pass levels. It 
does not give also the statistically significant number of repeat candidates, nor those who had been held back the 
previous year for fear of negatively impacting on the phenomenal increases in pass rates. Such test takers who 
gained university admission were, on the average, far less than the official pass rates and only ranged between 
27% and 33% from 2009 to 2013. 

The sample studied in this article was a part of this 27-33% admitted into a Teacher Education and 
Training Department for a Bachelor of Education degree in 2012. There were 153 students enrolled, all taking a 
content course in English Language Education (ENGE). Their content modules were ENGE 111 and ENGE 121 
for the two semesters. 200 other students were enrolled in the Faculty of Human and Social Sciences (HSS) for a 
Bachelor of Arts degree, and they were taking two courses, English Language and Literature (ENLL): ENLL 
111 and ENLL 121. The hypothesis driving this article was that assessment procedures, especially the form of 
examination questions, have a direct impact on how students study: performance indicators and grades derived 
from assessment certify what has been conscientiously learnt by the student. Again, Knight (2002:277) aptly 
observes that university lecturers and assessors in general look for evidence of achievement, even though he 
admits that evidence is in itself a teleological concept and that critical thinking, for instance, is polymorphous. 

Conceptual framework and literature review 

This article extends the discourses on testing practices and offers ways of interrogating ‘tests as disciplinary 
sites’ (Foucault, 1989). Messick (1998) together with Shohamy (2001), argues that tests, as ‘sorting machines’ 
(Spring, 1998) violate fundamental practices of democracy, especially when due regard is taken about the reality 
that tests lead to far-reaching and high-stakes decisions and conclusions about individuals (Bourdieu, 1991; 
Tollefson, 1995). 

Assessment entails the process of gathering evidence and making judgments about students’ 
achievements in relation to stated learning outcomes, and recording and reporting of these judgments. In 
universities, academic staff are responsible for the education and academic standards that they provide for 
students to achieve the qualifications they are awarded. This practice affirms the observation that assessment 
should both enhance students’ learning experiences and improve the quality of teaching and learning. In a 
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nutshell, assessment is an important quality assurance mechanism. 

Assessment criteria against which the stated learning outcomes will be assessed must be developed for 
each assessment task in each module. At South African universities, the assessment criteria for each assessment 
task must be explicitly articulated in the learner guide. 

In addition, assessment criteria should be openly and clearly communicated at the same time as the 
assessment task is given to students, both in the interests of transparency and accountability, and to enable the 
students to use the assessment criteria to monitor and develop their own learning (UN1SA, 2010:3). In other 
words, students must be informed as to the knowledge, skills, values and attitudes they need to demonstrate in 
order to achieve the stated learning outcome(s) of the task. 

The assessment of students should therefore be designed to achieve as many of the following purposes 
as possible: 

• To be an educational tool to teach appropriate skills, knowledge, values and attitudes; 

• To set educational standards; 

• To determine minimum levels of competence; 

• To provide a measure of student ability (an aspect of consequential validity in Shohamy’s 

(2010) terminology) ; 

• To determine whether students have met the outcomes of a module/course; 

• To inform students of their competencies; 

• To inform lecturers about the quality of their instruction and to foster ongoing development 

thereof; 

• To contribute towards the evaluation of a course; 

• To detect learning problems; 

• To contribute towards decision-making related to student progress; and 

• To provide students with feedback on their progress. 

One of the issues saliently raised in this study relates to how empirical research could establish with 
some degree of certainty whether or not students have fulfilled such expectations as outlined above. 

Shohamy (2010) contends that test developers, in this case university lecturers, have an agenda 
regarding tests: good tests possess features that provide indications that they can measure accurately what has 
been taught and learnt; weak tests fail to elicit responses that yield meaningful interpretations. . Besides defining 
what is worthwhile knowledge (Kelly, 1995), tests also serve to include or ‘exclude, gate-keep and perpetuate 
existing powers’ immanent in the test developers and the testing institutions. This gate-keeping role of tests 
breeds anxieties in test-takers who logically seek to know ‘the scope of the tests’ so that they access the 
privileges through narrowing their focus on the performance indicators proscribed by the tests. Such anxieties 
are bred from the recognition that universities are generally expected to have learning goals that are both 
extensive and complex: the tacit currency of universities is complex learning and ‘challenging’ essays are the 
most common device used to assess complex cognitive achievements (Heywood, 2000; Venkatakrishnan, 2014). 

The School of Teacher Education and the School of Human and Social Sciences from which the 
research participants were enrolled were perceived as the ‘sponsoring authorities’ in as much as they were also 
custodians of the necessary standards and confidentiality of the test processing.’ Since tests are confidential, 
Foucault (1979:56) asserts that ‘testing is the primary disciplinary site’, while Messick (1994:17)) adds that 
testing has to come to grips with consequential validity, that is, the consequences of how test scores are 
interpreted and the ways in which the tests are subsequently used. Thus, in perceiving tests as seminal to 
students’ progression and admission into prestigious qualifications (Ball et el, 2004; Canagarajah, 1999), there is 
a sense in which test takers view themselves as being at the mercy of test administrators and consequently as 
powerless pawns. 

With students agitating to know the scope of the paper, it is logical to affirm Shohamy’s observation 
that pedagogical instruction, together with the development of higher cognitive skills, appear threatened and 
compromised and narrowed by such student demands (Shohamy, 2001:107). Test scores consequently become 
inflated on a yearly basis since most teaching is narrowed to best fit the test specifications, as evident in Table 1. 
Ultimately, the validity of tests - in schools at least - reaches questionable proportions (Spolsky, 1998; Shohamy, 
2001). The tacit encroachment of the question on the scope of the examination into university sites bears heavily 
on what gets taught and assessed in the universities when the surveilant gaze of the students is focused on test 
content. 

Kurebwa and Mberikunashe (2013:146) analyse the negative washback of public examinations in 
selected urban schools in Zimbabwe and ultimately endorse the view that the explicit school curriculum is 
subordinated by the power of tests. 

Public examinations have both positive and negative impact which includes teachers adjusting their 
teaching to suit examinations. Teachers compete amongst themselves to beat the system...drilling pupils on test- 
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taking skills and promoting rote, superficial learning. 

In their study, Kurebwa and Mberikunashe (2013) reach the conclusions that ‘scope of the paper’ 
becomes associated with teachers focusing on the content of the tests, teachers administering repeated practice 
tests and teachers training learners in specific answers to specific test tasks. Against the warnings of Paulo Freire 
(1982) and that of Bowles and Gintis (1997/2008), such superficial pedagogical practices promote transmission 
styles of teaching and learning: memorisation, rehearsing and regurgitation. 

Research design 

The research design utilized in this study was based on a qualitative, quantitative and naturalistic design in order 
to identify and interrogate the research participants’ views on their experiences and the impact that high-stakes 
examinations at university level had on them. 

This research sought to unpack the meanings that first-year degree-level test-takers ascribed to the 
concept of ‘the scope of the paper.’ It also sought to establish what aspects of the test scope students studied in 
order to prepare for the test. In tracing such concepts, inferences were made about the concepts that the students 
marginalised in so far as they were perceived to be ‘out of the scope’ of the examination. This research was 
particularly driven by one participant who unreservedly intimated that she hated it when the lecturer did not tell 
test-takers in advance what the test would cover. This view reflected a test-taker’s anxieties and perceptions 
about tests as ‘threatening, powerful and authoritative’ instruments (Shohamy, 2001:13).When university 
students asked about ‘scope of the examination’, were they implying a more succinct but loaded question that 
high stakes assessments are marked by unequal power relations between assessor and assessed? 

Research question 

This research sought to establish what examination taking students had in mind when they asked their lecturers 
about the scope of the question paper. The investigation entailed identifying and interrogating the participants’ 
views on their experiences with, and impact of tests and examinations at university level. 

Participants 

The participants were first-year students registered for ENLL 111 and ENGE111 modules in the BA and BEd 
programmes respectively with two lecturers who taught these courses. The rationale for selecting first-year 
students was to establish whether being in a university academic environment where students do a lot of 
independent reading and writing and make their own notes would not be discouraged by the workload and be 
obliged to ask their lecturers the scope of tests or examinations they were to write. Students enrolled for ENGE 
211 and ENGE 212 were interviewed for triangulation, specifically to establish whether or not the same anxieties 
propelled these students into seeking the scope of the paper. The two course lecturers were involved in the study 
in order to identify and describe their experiences of students’ persistent requests for the scope of the 
examination. All participants were informed of the purpose of this study and they obliged to respond to the 
questionnaire and the interview questions without coercion or threats to their anonymity. In order to conform to 
the research confidentiality clause and ethics regarding disclosure of participants, this report invents fictitious 
names for all the respondents. Their identities are not divulged. 

Piloting 

A pilot run of the questionnaires was done to ensure that the sample of research participants would not have 
problems with what the final questions sought to elicit. 20 pilot-test questionnaires were selectively administered 
to BEd students in order to assess the responses that students provided to specific questions. The pilot 
questionnaires were also meant to improve the final questionnaire and to anticipate the response rate, including 
the pre-setting of analytic categories. 353 test-taking university students were purposively sampled to participate 
in this study. 153 were enrolled at university for a B.Ed. and 200 for a BA degree respectively. 

Data 

Data for this study was collected from three sources: students (through the questionnaire and interviews), course 
lecturers (through semi-structured interviews) and examination question papers (document analysis). 

In order to determine the frequency of the scope of the paper question and to establish the reasons for 
asking this question, 200 questionnaires with closed and open-ended questions were circulated to ENLL 111 and 
ENGE 111 students. Although these modules have an enrollment of 200 students in ENLL and 153 in ENGE, 
questionnaires were distributed to available students at the time of data collection. The circulation of the 
questionnaires was followed by interviews with the course lecturers and the exploration of module outcomes of 
the courses in question as well as the end of module examination papers. 

Document analysis, specifically of the four modules and the semester test papers, was an on-going 
process of this research. Module outcomes and examination papers of the two modules were checked to establish 
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how much of the outcomes were covered in the examination. Internal and external moderators' reports on the 
scope of the university test papers and their admissibility as accurate gauges of specific competencies and skills 
were analysed to establish their accents on suitability and complexity levels. Sample examination scripts were 
retrieved and analysed for their depth or shallowness as measured by the marking memoranda developed for 
specific test papers. 

Results and discussion 

In this segment, the results of this investigation are presented in three tiers: results from the questionnaire, the 
interviews and finally those emerging from document analysis. 

Student questionnaire 

Out of the 200 questionnaires circulated to the ENLL 111 and ENGE 111 participants, a total of 152 
questionnaires (80 from ENLL 111 and 72 from ENGE 111) were returned completed. In both courses the 
majority of students (89% in ENGE and 76% in ENLL) affirmed that they asked lectures for ‘the scope of the 
paper’ and only a few (11% and 24% respectively) in both modules responded negatively to this question. Some 
of the reasons given for requesting the scope were that the amount of work covered in the modules made it 
difficult for them to know exactly what to focus on when preparing for examinations. Also, they needed to know 
where to focus on in order to save time to study other course modules. Others wanted to be given guidelines on 
the type of questions to expect because they had no experience of writing a university examination. Most of the 
students are products of an education system that promotes drilling, more so for examinations. These learners’ 
request for ‘scope’ comes as no surprise. As these students were first-years, wanting to know the kinds of 
questions to expect implied that they are not aware that previous examination papers were available on the 
university website. For those making this request in order to focus on important aspects of the module, this could 
mean that they either did not know that module outcomes in a study guide provide guidelines for 
examinations and tests or they simply chose not to use the module outcomes as their focus when preparing for 
examination. 

The exploration of the ENLL 111 Unit 1 module outcomes on literary texts showed that the students are 
expected to: 

• read, think and discuss poems and short stories, with particular attention to aspects of 

language, form and content; 

• consider whether reading literature requires special reading approaches, skills or strategies; and 

• come up with a personal reading strategy for literary texts, particularly poems and short prose. 
A list of questions on reading, thinking and writing about a poem were provided as guidelines to enable students 
to focus on important aspects of the poems and stories. But an examination question such as the one below from 
a 2013 ENLL 111 question paper requires much more than the outcomes listed above. The focus seems to be on 
comprehension and analytical skills not covered in the module outcomes. 

By a close reading of the following poem, say what the poem is all about and 
how the speaker in the poem is conveying the main theme. In your analysis, you are 
expected to focus on poetic devices such as metaphor, imagery, diction, rhyme 
and rhythm and any other linguistic or literary features you may observe at work 
in the poem. [50] 

This type of question is totally different from the ones in Matric exam papers and what compounds 
students’ examination anxiety is the fact that some were studying poetry for the first time. These could be some 
of the reasons for them requesting ‘the scope of the paper’. 

Some of the reasons for not asking the scope of the paper were as follows: ‘as students we need to 
understand everything the lecturer taught and not study for the sake of passing an exam; I did not ask for the 
scope because lecturers set what was taught in class; it helps me to go an extra mile when I’m studying’. The 
reasons given by these students show that they need to prepare for everything they are taught and not just focus 
on aspects covered in the examination. 

To the question which required the students to specify what they expected from their lecturers when 
they ask ‘the scope of a test or exam’, 98% from ENGE 111 and 78% from ENLL 111 expected their lecturers to 
provide them with possible questions and possible chapters on which the examination would be set. These 
percentages reflect that most first-year students expect to be given guidance regarding examination preparation. 
Very few students misread the question and said that lecturers who give students scope do so to have a high 
number of good performing students. 

In the ENGE group 72% indicated that at high school they asked their teachers for the scope of the tests 
and examinations and 28% pointed out that there was no need to ask for the scope. The reasons given for both 
positive and negative responses varied. For example, some of the reasons for the positive responses were: 7 
wanted to know everything about what I was going to write; our teachers would not refuse to spoonfeed us, they 
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wanted all of us to pass.’ Some of the students who did not ask for the scope indicated that their high school 
teachers told them what they should expect in the examination; others knew that their teachers were not the ones 
who set the final examination and therefore read everything to prepare for examinations. 

In the ENLL 111 group, the difference between the students who requested their high school teachers to 
give them the scope and those who did not was insignificant (54% positive response and 45% negative 
response). Those who asked for the scope indicated that making this request was just a formality as their 
teachers always gave them the scope of a paper. One student from Zimbabwe indicated that she would not dare 
ask the scope of a test or examination because their education system discouraged this practice. 

Most students felt that being given the scope of a paper at university level made it easy to study for the 
examination and get good marks, but also expressed concern that studying everything that has been taught was 
cumbersome. In order to ease the burden of studying everything, they expect some guidance with respect to a 
narrowed focus on examination preparation. 

The overall students’ perception on the scope of the test or examination shows that they need to be 
made aware of the importance of the module outcomes in each unit in their study guides. It is assumed that at the 
beginning of an academic year new students are taken through the module outcomes and learning activities to 
make their work more manageable and to avoid examination stress and anxieties. An interesting subsequent 
study could strive to establish how many lecturers make this information known to their students. 

This research tentatively established that in asking for ‘the scope of the examination,’ university 
students were, in essence, subverting undemocratic testing protocols. The intention of such a question was to 
gain access to ‘legitimated knowledges’ as set in the examination questions: the university students sought to 
participate as active agents in the ‘test democratisation process’ so that the power of tests (a' la Shohamy, 2001), 
was transferred from academic lecturers to local sites, i.e. the university students. In its apparent naivete', the 
question posed by university students advanced the notion that assessment needs to change from an impersonal 
‘sorting machine’ (Joel Springer, 1989) into a platform that generates new insights about learning and outcomes. 

Student interviews 

Relating to meanings ascribed by students to the ‘scope of the exam’ 

As mentioned earlier, ENGE 211 and ENGE 212 students were interviewed in order to corroborate the 
experiences of university students in the area of examinations, anxieties and individual approaches taken to 
alleviate such hindrances. There were two extremes that emerged when this question was posed to both ENLL 
211 and ENGE 212 students. Kaogelo, male, indicated that he wanted the ‘lecturer to tell me what questions 
were set for the exam. This makes me focused. I read with the questions in mind and prepare my answers for the 
day. ’ 

Rethabile, female, was enrolled for ENGE 212. She indicated that ‘scope’ includes ‘just a general idea. I do not 
want exact questions. But I also do not want to read wildly. Sir, if I know which books are going to be tested, 
then I will read those. Time...eish. I have other modules and if I spend all the time reading all the prescribed 
books I will not finish. So I want just an outline...a guide on what to read so that I get at least 50%. ’ 

Interspersed between these extremes were indicators that students apparently preferred lecturers who told 'the 
truth’ about the content of their test papers as opposed to those who ‘ambush us with unexpected questions that 
are usually difficult,’ as mooted by Karabo who was enrolled for ENGE 212. Clearly, the ‘culture of 
expectations’ is apparently a pervasive one. 

Relating to awareness of the limitations of preparing examination specific answers by students 
Babalwa, registered for ENGE212 gave the following response: ‘We are here to pass. And we pass when we are 
prepared. When we know what the test tasks are going to be and we discuss these with others who are more 
intelligent in our groups. ’ Whether there were inherent limitations in preparing for the exam or not was not a 
point to contend; Babalwa would rather participate in examination skills honing than waste time reading for 
subject matter knowledge and pedagogical content knowledge (Venkatakrishnan, 2013). 

Bojosi also added that ‘when we know what to expect we read around. What use is it, Sir, to read what will not 
come in the examination? If you know a lot and then they do not ask that, what do you do? You fail, sir!’ For 
both Bojosi and Babalwa, disciplinary knowledge and depth of knowledge are peripheral in their educational 
endeavours. Success in examinations is privileged more; so is the ultimate evidence of that success in the form of 
a university certificate. 

Relating to reactions towards lecturers who did not give the ‘scope ’ of their test papers 

Students revealed startling notions about lecturers who did not reveal the scope of their test papers. Ngakanyane, 
one of the participants, outrightly indicated she ‘ hatejd]’ such lecturers. Thamsanqa indicated she was 
‘concerned’ when lecturers withheld the ‘test scope’ because ‘then they plan to fail us.’ For Thamsanqa and 
others, the test should never be an ‘ambush ’ since ‘we are the students and should read what comes in the test. ’ 
Probed further on whether such an approach to subject depth and rigour did not compromise the students’ grasp, 
a significant number indicated that as long as they were in the loop about the constructs tested in the 
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examination, they would prepare accordingly. 

Xolisile’s response to this question offered a more nuanced perception about her understanding of the scope of 
the exam and the positions taken by different lecturers: ‘We write differently, even when we know the question 
beforehand. Check our assignments. Even when we do these, some copy from others but we still use different 
words. If someone writes well, then they desen’e to pass. Why should she hide the examination? I get scared. The 
best lecturer is [X] and we always know what to study. ’ 

Course lecturers’ interviews 

Relating to what lecturers reveal to students about their testing instruments 

The ENLL211 lecturer interviewed revealed that she consistently advised her students to focus on the critical 
outcomes of each of the modules that she taught: ‘I tell them that the scope of the paper is what I have covered in 
the module.’ This admission by the lecturer was also evident in the ENLL question paper for 2013 in which the 
material in the study guide reflected both the content breadth and the cognitive levels that the course sought from 
the students. 

The ENGE 212 lecturer indicated that he was ‘uneasy’ about questions on the scope of the examination. He 
indicated that if he asked students to read broadly as suggested by the module in order to demonstrate the 
intended module outcomes, then the greater number of students would under-perform in the examination. 
Notwithstanding, the lecturer was also aware that revealing the ‘real scope of the exam’ would compromise 
confidentiality and even allow the students to rehearse their responses. 

Relating to lecturers’ experiences of students’ responses when the scope of the paper has been divulged 
Asked on the strengths and weaknesses observed in students’ written responses, the lecturer for ENLL 211 
indicated that answers showed marked differences: some were strong, convincing and articulate while others 
were phlegmatic and relied heavily on regurgitation of module content without selectively appropriating such 
content and marshalling it to answer specific questions. In her own words, ‘it depends on the questions set, the 
level of the questioning. Lower order questions are more easily done than challenging ones. Some students 
memorise and regurgitate answers to the extent that there is no connection between the question and the 
response.’ 

The lecturer for ENGE 212 indicated that there was a significant paucity in the quality of responses, especially in 
the essays. Arguments were under-developed and the breadth of consultative reading was minimal, suggesting 
little if any extended research beyond what the module offered. Questions demanding analytic and evaluative 
skills were less taken than those that elicited descriptive and recall cognitive levels. 

Relating to lecturers’ efforts towards discipline-related depth and breadth in module and course delivery against 
the strictures of test papers 

In terms of the lecturer’s efforts to provide sufficient discipline-related depth and breadth in the course delivery, 
the ENLL 211 lecturer divulged that ‘student performance varies, naturally. Some are very good and go beyond. 
There is evidence of extra reading and integrating skills, coupled with individual analysis. Others stick to the 
modules and workbooks and reveal shocking gaps. Here, the memorandum guides my marking. There are 
performance bands that clearly tell where to place the student. And the external examiner generally agrees with 
my assessment.’ 

Document analysis 

Relating to selected moderated examination scripts and marking memoranda 

Six examination scripts that had been moderated were retrieved for content analysis and also to establish their 
conformity or deviation from the standards set out in the marking memoranda for the respective courses. The 
hypothesis driving this exploration of content analysis was that if students had been made aware of ‘the scope of 
the paper’ prior to the exam, there would be some tell-tale evidence of regurgitation or rehearsed material. 

Two scripts that scored 60% and 68% in the examination demonstrated sufficient depth and some 
sophisticated analytic rigour. They provided independent opinion and supported arguments. There is no way to 
establish that the students had prepared for the same ‘scope of the paper.’ Whether this was coincidence or 
collusion, the two candidates answered similar questions in the examination and the quality of their answers was 
comparable. 

In contrast, there were 4 scripts that had scored 34%, 38%, 40% and 42% in the examination for ENGE 
212. These answers were largely incoherent and apparently reproduced module content without adequate 
interrogation of the questions set for this examination. Based on the quality bands set out in the memoranda, 
these scripts fitted the lowest band insofar as they displayed inadequate content mastery, inconsistent linguistic 
expression and presented material that did not respond to the questions set, stricto sensu. The first examiner’s 
mark, who is also the lecturer responsible for content delivery of the module, were confirmed and ratified by the 
second examiner with no changes suggested in all the six scripts. Again, it could not be established here whether 
the marks were an exact fit of the quality bands of the marking memoranda or the apparent agreement between 
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the two examiners was a fait accompli. 

Discussion and recommendations 

There is evidence emerging from both students and lecturers that ‘the scope of the paper’ apparently implies the 
test content, the format of the examination and the specific areas that the students should read around. For a 
majority of the research participants, revealing the exact details of the test would be a boon. Students apparently 
have abandoned pursuing the rigorous curriculum specifications and content. Rather, they train their scholastic 
abilities on being test-wise. Evidence suggests that university tests have lost some degree of their consequential 
validity since the test-wiseness of the students marginalises critical learning outcomes at the expense of ‘surface 
learning approaches.’ There is an apparent compromise in test validity emanating from a significant distortion in 
the kurtosis curve in university students’ performance across modules and courses. Whereas second examiners 
and external examiners come in to ratify scores and grades without knowing the identities of the candidates’ 
selected scripts, there is some ambiguity relating to how much university candidates have rehearsed their 
responses to examination questions prior to the examination because they have been admitted into ‘the scope of 
the paper.’ 

From the findings established in this first series of research, the importance of module outcomes and 
learning activities in relation to tests and examinations should be explicitly stated and explained to the students 
during the first contact sessions. Study guides too should have sample examination papers, and perhaps some 
model answers to assist students. Rubrics would further qualitatively enhance the utility of modules by offering 
grade descriptors for typical examination questions. This way, the university in South Africa could 
authoritatively discontinue the practice of lecturers having to respond to students’ persistent demands for the 
‘scope of the paper’ since guidelines and resources would have been made available. 
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